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[Rffll I BR^S^v^T^^-— — nTSB*4EB 

e>45 7^7 f -^»^ftilJt5 (S101-S11 
0) 0 ttt, Ifctti^ttfc^U— .^^jjr^ov^T, n 

1*?—* &m®rf<> ( s 1 0 6 , Sill), ;H 
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I 

I— — 
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Si 05 " _ " 
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SI 06 



7 1/-^,^ 




?f-^*B«a —sin 



»B«»*«fc«»S*ufc«»©7 v — At* — 9 X *) , 
ttvx&ft&tixte*)^ 

tWEJ4ttJ#Rtt. fit 7 

[»*«3l fJfHteMT*— ^fiGP SJcJ:oT#^tL 
fc fe G> T* £> 5 r £ m^jg 2 [CfB^^M^M 

to -c fc 5 r & £ flHR £ i~ s Hfs&g 2 fclB*<&M«8y£SS 

So 

[»*«5i ffte«i«^ft». atriaa*^^— at* 

ttXfmZtlX&Q* 

t*— *a>£/«£5 :7 At*— *p£»£fc!i-5 r £ 
(-fx, 

^I^TTfeofc^C^g20^)7 I- At*— ^&' 
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o] 71/^7-^wt^yy^RB^ 

^0J:^2^^j»JiJC^oT»»$nfc7W-AT*- 

20 K2^W©*^±fr#ft^^#7 i^— At*— ^Sc^3»ttl 

ttBR##*T*f»Ufc*»©7V— At*— 

# *r»r*-5IB5l #*« t £Mfc<{i;t 6 r t & 

^Hfc^W— At*— ^Srttffl-rsattlXfii:. 
30 BtrBttfflXS-CttUi ZtitzMm-i- -5 7 At*-^ SrfiiJ 

^-rsgy^xg^, 

WBB'j»xa^ii?T«n:3Sofc7 at*-^^, mm 

1 3 ] «rB«»¥a^tt, buI51S^co^ 
At*— ^O^^tL^igjK^ffiaS:^fifiT*-^d5 

StrEtttBXStt. ffi«T*-^365— 8^7^-^^ 

40 3g^-& 0 

[W*«l 4] ItrlSffillT*— ^IJGP Si:«3;oT#e 
5] StfffiM«T*— ^tt-t^ttftfts^i^ 

50 ^"fe 



3 



(3) 



&m 2002-269592 

4 



i e ] «HB«»^a^tt. tfnasfta)? 

[»*ai8] SMBiMSXgWu 2o(D7Wisr- 
[»*919] ftrlBW^Xatt, 2 0(07 1 At- 

Z^b&ftWLb'TZ^b&Vfmb'rz&li&Ml 7fcfE* 

1 7 75^1 9©V^-T*t*^lB««>M««»6ra. 
K«+SK3EXS*3EI-«*.. 

itBR^xa-T?K3es^fc*^» v^raw^ffisi-s 
b-tzm&m. 1 3 ^ett^>H*«L»£rtt. 

[»*«2 2] 7^7^(Dt^D ^KHW* 

^m^^m-r^^MTMb, 
mm2 t ^m(Dmmbmm^>y r ]) ^^m^m^^x, 

■f ssuxm^ 

ffiTffiK ffl XS-Cff a S Jxfc. 7 V- A * ft t ft 5 «t 5 

r^§i * *nnirz>wfi\ % is* zm^m^z z b 

2 3 ] fS*^ 1 2 75^ 2 2 <OV vftLJWlIlE* 

uw7*#7j**tei&^zzb*&mb^ttw&&. 



2 4 ] 1 2 2 2 G> V >-f *xa>miE« 

[^WOi¥ffl&f&91] 
[00 0 1] 

[0 0 0 2] 

jtsnotrWBu m3&ztntm&mitf s -*&t>b\z^ 
w&vttmn&w&ftwm&m^x&M&rtib u-t^m 
-rz&&&mm£inx^z m^amm. «-uj> h«\ 
am. mm. ww : tt »ib**i*Mfe* *9*m *fc«pt? 
£m<Dmmmk4b{z-D\,^x" m^y^^T^, pa- 3 

-4, pp. 276-277, 1997^) . £fcfc±K*& SBa* «yiU 

>t"Uj. ffltt: "^ib^^^;^ ^^rffiv^Ttmr^ 
WMBttffllB©*!* (2) -JWMftMv^tlEfilBa 

20 ^*:fc£t, pp. 67-70. 1997^) Q 

[0003] &mmz»i££titcm£mm\z£<>xm£ 

m&mb isxmm-tz^mb urn, m^m&y*-?* 
hb\z^m^K(o»fm^^^n^^ ^^<ocg 

mts^tcwm&ftfr-riz^ n^m&zm^xfom&m&m 

Image-Based Rendering (I BR) Stfffj&Saff^a 

BR&ffitem^mmzm^^x^ztzi&iz, ^hw^mk 

[00 04] ^©ipftlBRSjfitffl^T?*-^^ 

5o ^(Dtzfr, z\<Dm<Di/x7"j*\z33\,^xn, %?*m& 
"T—*(D%r7 \'—j*bm&imftv>®m.b zttfeftttx 

40 60 

[00 0 5] SUgRgrt^fiif-^ *»S#i5fei: LT 
tV^GPS (Global Positioning SysLem) (^f^^^ 

^T*fe^ 0 GPs4^e>#e>^fiiT^i:, ^ 

ffi^^Tm#lto#^$|^^^TV>^> MMBTl 1-168 
754) o {i®^-^ l^-a ^tL^ &fMir — 

*b, m^m^- $ <d&7 \s — j\ tzttM ufc ^ a ^ 
50 -h'b&ttfcttttzzbx\ m^m&7*—*<D&7i — 



5 



(4) 



&m 2002-269592 

6 



[0 0 0 6] ~(D&of£fe*g&m]ft<DV* — ?*;V—\z. 

Bi«SrSJ?>ttiU rnsriitsr ir^#^^tt5o 

[0 0 0 7] 

[0 0 0 8] aWM»4±|B©WH*!:IK*-C*S*Lfcfc© 
[00 0 9] 

[0 0 10] ±IB«>B»Sra^-fSfcaC>^**M 

-f&mm-rzibmjLmk, mmt&mjimxi&mztiti 

[0 0 1 1 ] 

[0012] <mi<onimm>*'r. *$mmm^x » 



^-r—^ri^^y' ^-^IH&x— *S:±j5fcl^ ZV*;? 

[0 0 13] iaiW:*ISifi«18lJ:J:-5^^~^^/u—> 

IBM^BK l * Sr-g-Az-CfllriS S *t£ 0 0*7* l 
4fe • WMlt^3 0, JfcfftM 0, 

5 o , m*s±mm 6 o , 7 o «r*i- 
[ooi4] m*&f—*&#n i o ^-cPxBi-^M 

RAM*fctttt©^«E1KI611ic{IHlFS*L-5 0 
[0015] ■« - *H#*Wlt» 3 0 EMfcr— * 

fc<s#£*LSo ^\ ■«-»B#«ftM3om 

^5 0i:^ 9 ttiM-SJ:}^ GPSffifStSft 

^7w-A % gp sKtn&^mzfiZo 

[0 0 16] Sf^§P4 0«. * — h\ i/s 

[0 0 17] W&W*kfflWU6 OWc. SWHB4 od^<z> 



tilt?*— ^^^MMftr-^^tilU ^g&7 

[001810211 Blft^-^ft^ffil 0l^#£;h, 
5 7 W — A * £ JBtftf- 5fc£)<Z) IMfey*— ^ ifc^ v- J* 

tc, C^PHfe^— ^ifc^ls^^A 9 0 H ®^£f$9 l , 

5 0 £*T> #SP^ov>T, H 3 — H 5 U-CBJBir 10 

[0019] H 3 H S^SFP 9 1 ^*fifcS:B*Bfc^-f ^ 

(n^l) (Ot*rt*>7 (91-1-n) £ 
9 1-1 ~~n(Dnt:<DM&*4 S >^^r-i^^ii:TVN 20 

[0020] m4ii N ib««b9 2<Dmi&&mMiz7Tk-r7 

py^8^5, IB««9 2«^>f As— K3B£»9 5 
fli;i5 0 *^§15 9 1 (Dn-£<DVr&# * y 9 1 - 1 ~n 

frt>(om&tiizi^ ^ti^titsmn 9 2 - 1 ~ n <oxt) 

^-T^-r^a— K«r#VCR 9 2-l-n(I«t6 0 
VCR92-l-nH MfciS Vy**** yfrb<Dm 30 
&A£)k 94 An — K»£fflS9 5^^^^A^- K^r 

[0 0 2 1] H±©i5IIlUTVCR9 2- 1 ~ n 0>*r 

fi6Sr»«Bf^-r^ci ytmX'thZo A/D&««9 3 
tt. /^yt/^yf a ^-^ ®TPC) 9 6ir. VC 
R9 2-l~n©*^^t-*^7 ? :3-- ^-^^ . 40 
*-K (KT, *t^f#-K) 93-l-n^t 

*— K«r**urt>J:v\ A/DKft«9 3tt> =&VC 
WJttfAVI7t-7yh) fcKKlU PC 9 

[0 0 2 2] JSfc:^ *f*^Afc:}3^Ttt. ^ An — 50 
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GPS 9 7j:0filf^^#U ^bfe&a^- 
[0 0 2 3] ^6 ttlBfcT*— 1 0 Kfcltfc trf 

a-r—ft&w?*—* (GPsnsBif-^) 

* ^ A a - KaMWm * ixfcteS^- * £ jftMftM 
9r (nil ft) 36s#ffii-s 0 

[0 0 2 4] fc*5, GP SKjottS 3»C7c{4®<D»tfi« 

^e>, GPSCJ:oTir#^iS, ftfl^-^P 
(0, S:*^ A=x — K*i(B7 2^^^^-T An — 

[0 0 2 5] 4*5, mtef—fit**^* • *-K9 
3-l~n^ICTPC96 9 Blfctt. 

[0026] mwHE3&w 1 *-ov>Tttw-rs. 

■r— ^Hx^i^^^A 9 0 i:J;oTl|f)tifc7V^7- 

5 tiii^iif&x— 1 0 <D3*ftP>i% ® 1 
Ufc^^-^«#gl5 2 o. ^iSW-rt^— ^«#»5 0 

[0 0 2 7]CPU10Ht V*<<X? 1 0 5^fcfiR 
OM10 6. *fcf±^HWE«3fi« (WW lEWSft 
TV>5^n^^A^^fT*r^^i:icJ: 19, m&v*-* t 

[0 0 2 8] C P U 1 0 1 fri&7jk 3yhn-7l02|: 

ttvx&m<om^t%7F&ftozt\z£ to y $,fayhD 

— 7 10 2*5j;t/7 A/<y7 7 1 0 3i:<tot^/T 

*i 0 4\zmm<om&&tj:£tiz> 0 4*5, 0x1^^ 
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yhn-7l02^UCRTC, i/7l§10 4^tT 

25, CRTC102, 7W^ 7 77l0 3MCR 
T 1 0 4 fi> ±m<K>&^to 7 0 Sr«j«i-5 0 7^10 

m&m&&m • i -^^-if^f^A^^tT 5 
[0029] rk:, &±(omatim*.it*3am*<Dir 

[00 3 0] Hf*?*— 10 Kltt. _b^Ufc®^ 
^42ft^7^9 0trJ:oT, n^tf^^ 7 
9 1- 1 — n^ioT^fettfctrx^^— ^^S<3< * 
^3 — K#^>7i^ — At* — $k^ GPS97i:j:oT 

[0 0 3 1] Sift • JttH*f-itf+tt« 3 0 W— ©#>f 
f£rrz>kkh\^ «jH7*—^fil»«2 0fc«»S*tTV> 

X— ^ 2 10 U ^WltT*— 5 0 fdfe 

[0 0 3 2] fc*3. ^f+tt^— ^»JA«5 Ofc&W 

rcoi D«r»*-t--5 7U— Afc#Jpi-5 0 loco it 

[0 0 3 3] Hsjir^flH^^RW-rsH-cfeSo H9 
k:*5v^t, 08*. tf i d^sc l-efcSECd^i c 2t?fc5 
K»j»^i**ixfcj|»»*cR 1 fci^5 I DjWMF-SJxtv* 
So r*b£> I D*«Hfc i£I^X--*{£#S$2 

[0 0 3 4] GPS f-^^^I^VNT^MC 1 t C 

i:^nfc7V-^»^i|R 1 fc#js-rs c k kteZ> 0 
^tt, KdAc l i c 2 lc#«t57 V'-AKtttft 

mci, C.2<OIDm$^ 7W-A»©*7W 
— MZ\t^ itlCR 1 - 1 -R 1 - n W4$tt5o 

[0 0 3 5] fc*^ E»Si:7l/^iO#Mmt 
GPS ^->? \zft-z>x£M&)\znt>tiZh<Db Lfcj&S, 



[0 0 3 6] ^(DXolZVX&mZtlfZttfetttfT*- 

**«7 o^^-r-<#[iift^^#jtT f -^ 2 1 

$15 6 0#LhsE£>J; 5^bT^®«<75^^^, 3c^«7 
0— <7?^SrfT5w k\z& fr*—*;*^— 

[0 0 3 7] gUi^^^AfCjoV^T. M&T*— 

:r-*<z>ifcJfe£fT5 e Sot, HBSI«r*fTLttas&* 

[0038] *mMBm<om& • mrnM^nn^ 3 0 
[0039] MSi^J&s-ett, m^-^^i/^T-A 

9 0f^J:oTJ(X^$tb. ®«l^-^^:#$Pl 0fc^#$ 

[0040] hi onnimmj&mzxzm^zmm* 

tdi5V>T > ^ ^ ^ ^ *m<D#LWy*— * (WT. GPSf 

50 -*^v^) ^ti^ W^ffi rij ^Kft-rs. & 
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{Z^y^fS 1 0 2 f^ioVNT, i#IOGPSf-^P 

i &m@L7 : -*&&m oxvwLmTz dr#fc>*>. p 

fc^t^T, :<DGP Sir—* P i 

gps * zm-r j £ r i + 1 j . -r*fc>*> 7 1"— a 

i (D^co ~7 \s* — A j (^fSGPSr-^ra^t 

[004 1] ^v^^S 1 0 4T% 7 A j £*fj£-r 
5GPS7-^ (-T^^^> j #g<OGP S*r — Pj 10 
&m&7*— 1 0 <fc OMU ^f^S105 

r-PitPj &Jt«i-5. ^ur, w#3fi$»uv^ (p i 

= Pj) M^-n, ^T7/S10 6^I^ 7W-Aj 

GPSf-^^-^ym 0 ftx, ^77^si 

loiDilU XTyZ^S 1 0 8T j &n&LTXlbtit£x 

«B*ttfc7U-AjiS5o (I 1 - I 5£i"5> B$cL 
TV>5^£\ ^tlb5oa)7W-AOGPSf-^ttf 

x^ry^fS 1 l l^mfrZo 
[0 0 4 2] — ^\ 1 0 5K*5^T. P i £ 30 

Pi^fL<iv«^tt, ^r^^Sl0 9^il^ i 
j {-f^A-r^o ttT, >>^S 1 1 0TM = 
nX^lttlf^T5'7'S 1 0 2^I6 0 ~ o LT, nffl 

7WA^^^7Wi»S^|Et5^ #£EL 

[0 0 4 3] 7t^^S 111 KiSV^-C, 

^^O^^ttfcGP Sf-^ (*^|3^>f A3 - K) 

»6t5^T©7u- *&fi«K*i-* 0 :Ht, ito 40 

[0 0 4 4] d 5 LTlSt5 7 V— Aj&Sglj^^ttfe 

[0 0 4 5] ta^ ±Eftk3ajci3v^, [§i-fcfE?>t>co 
(^r^S105) dS. GPSKiJ:S«-*»SS«K*r# so 
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tf>^&£\ s^S 1 0 5tC:j3V>T, 

f (Pi, P j ) < a 

tt, j»«^MS-e^4bStt-5GP Sf-^*Eif^» 

fig^-C^e^-C, ttW*«»i-5, w^J:9J--rtu 

S©io(07v — A^$to:im5o 
[0 0 4 6] £Lb<E><fc?fc:> *i«>IBI6»»z:J:Jxfi, 

tro^T, 1 o£> 7 V — A (5fc^<o:7U — A) ^LT 

[0 0 4 7] <»2^16»ffi>±SBSliafi«1l8"r 
ff)GPSf- ^^ori^-ffiB^ie^^— 

[0 0 4 8] fc*5. I2^M1X^ B5I^Ufc 
*i&*i/:*^A9 0GDA/Dg^g&9 3 \ZL&\,^ 

gps 9 7o«fc9&auc«-*«Ksn, ^tf-ai 

[0 0 4 9] HI ltt»2HJK?K«^J:5B0§l#*QtSSr 
Rit57o-f-t»h"C*>5o *f^^S20 1 

fcMJ&'TZ&Bty*—*) V i SrBfc^— 1 0 X 
9&#-T£o £fc, X^y^S 2 0 3^*5V>T. r<7>iS 
Sr-^V i ^^it^^*^*^**^— ^«rSi" j 

[0 0 5 0] **ry~?S 2 0 4T% ^ A j bttfctt 

^=rvzrs 2 0 5T'V i t>V j t> ^ ^C0T^5 
^tt, ^^^^^0 (V 
i=Vj=0) (DWr^n, ^T5/yS 2 0 6^I^ 7 

u-a j SrgiJss^-S'<<5aA7 f — j 

-<< j Srlo^IDSL, ^^^^S 2 0 8T j ^nJ^T-C 
htlYlX'r *y7S 2 0 4^IOo 

[0051] m&^—?&o<ommrz>7i<'--j*x~n. 
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tf> 5857^—^^0 071^—^^5 0 (11 — 15 

o) iSJRLTV^»frKW:, I 2— I 5 KHUIfc-^— 9» 

M^tb^r £Ul*5 0 ^T5'yS208Tj^n 

Xf ^7 P S 2 1 l^it£>5 0 

[0 0 5 2] — ^t^s^S 2 0 5^ioV>T. V i = 
V j =Okteh%:^m&te, ^^7^5 2 0 9^1^ 

i <DiB£r j KftA"t-3p -eUT, ^Ty7 , S210TM 10 
= nT^im^f^/S 2 0 2^ o :HT, n 

[0 0 5 3] T.'ry-fS 2 1 1 }^ioV>T. 

[0 0 5 4] £Uh(DJ:5K:, l2(^^«IiaW> 20 
[0 0 5 5] fg 1 Ol£K?»1l8"CKW UfcG P S 7* 

iotf-^sftwffiifta^t,^^ gps 

[0 0 5 6] ^fc, GPS7*— *MM6S:*tf^iBtt:&s 
fcS©^ ISt^7W- A«>ttai»*Sr<fcf!)fRlJii-5 30 

m^— * (omiffr h m— &e-cfc a £ $ ^fc 7w 

[0 0 5 7] <*3jBBfc«>ft*3* I2^1W 

-mp-fis©7w-A(oif ofcu-ctftau a 
[0058] i©8^, 7 u— a i ^EBBW-SHitSrP 

[0059] BBttM^fci^tt, mi(omm 

I Pi-Pj I < a 
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KtHM&P i> P j i:U«V^©-C, JilBsS-ett* 

[0 0 6 0] <fg4H**lB>SB43afi«ll|-ett, GP 

6sst5 7i/-^?rtiim cn^riJj^r^o 
[0061] mi 2&%4nmmmc£z>^\%%m* 

tf^r^^S3 0 1^i3^T s fsjx^#ft^7V- 
A^m-Ti^. «BHfl[ r i j (cl^t^ e ft^^^i/^ 

5 3 0 2tCl:}o^T > ^.^>^7)/ — A i b.(Dyti)fc(Dyti'$£kt£ 

6 7wi,^tj^ ri + ij , irt£t>*>m<ommz. 

[0 0 6 2] ^"ry^S 3 0 3 T% 7^ i i7W 

%xt, 7wa i^i^j »w— ftMoymm^&zkWft 

XT^/S 3 0 5 t7 j 

0 6T% ^7WA^i^^< j SrloiflgU ^ 
f5'7 P S3 0 7tj ^n^T-Cfcttf^^s/^S 304 

[0 0 6 3] — *\ ^ry/S 3 0 4 fCS^T. liE 
d5BMtK±-C*>ofc*&», 7 V-A i j ia.,oT 

^A-TSe *LT, XTy7 A S 3 0 9TM=nT^(t 
tl^T^7 P S 3 0 2^M5o ^3tT, n^^TCO^ 

[0 0 6 4] ^fy^S3 1 0t£33^T, ^— 3r 

S 3 1 1 t^joVNT. 4&<DZ* z 7fab&btltl'7 
^o^Tt), y^S 3 1 O-CHOIftSttfc^W— J» 

[0 0 6 5] &:ib\ »4|Sft^ttlC:^Tfl!V^:7W- 
3^£n<5 n 7^5 *><D— 

^ ^#ftfc:*fL-C 33**9 0° *>*ia?*r|Rl^fct>c> 

[0 0 6 6] ^fc. ±IEIIjte^8lT-Ji— o^^7i^ 
©7W- ASrfflv^^s. m®:©7 A^^ij(c:ov>T± 
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[0 0 6 7] <^5^^>^4Hl&^-ett[^-lil 
^T*$)5^^3W« (^7^^5 3 0 3, S30 
4) ^isv^T, S'h— fHigM£/3v^:: 0 U^Uft^, 

[0 0 6 8] mi 3\*m5mMmn^£z>m5\z&m& 

^■r^^^^S 4 0 1 frjoV>T, fsy^»ft©7l/- 

s 4 o 2 t^v^. :©7w a i t<oMtt<oM^tt£ 

[0 0 6 9] ^T^jy^S 4 0 3T% i«!:7l — 
WW4fe^ffl*fft*ffl:«H«Sr*«)5o ft*>\ WA^fe© 

[0 0 7 0] ^T7/S404m Xfy^S403 
i Stt^ j &m— &W(Dm&XfoZ bWmZ^ ^Tyzf 

s 4 o 5 x? u-j* j zmmmm^wfrbmm-rz^ 

"C j ^n^mtl^Ty/S 4 0 4— M£ 0 
[0 0 7 1] — ^ Xxs^S 4 0 4 ^jol/>T. ffl*fE 

&V>) 7r^S4 0 8^ i 

j ^ftA-f-So ^LT, 7ry/S409Ti=nT^ 

ltn^7 1 y7 P S4 0 2^IOo r5U, nfi^rco 

[0 0 7 2] #Cf^ ^t^^/^S 4 1 OKlfeSV^T, — ^ 

S4i -te<^^^^^e>#^ixfc^ 
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£ H U 7 u— A#-J§-g£i, S3 ^^A3-K^7l/-A^iJ 
[0 0 7 3] <JB6SdlS**>±IB*l-*5jOIS»« 

[0 0 7 4] JB6*tf*lTettJE^ 7U—J*%, US 

^ST% f (Pi, Pj) <a, SV^tt 
| P i -P j | < a 

[0 0 7 5] rcO<t 5WRj*i-5w £T\ 7V-—J»(Dm 

(a) ^i"5-£^-C#S 0 r<D«*. ^J*^ 

[0 0 7 6] ft 25, a©»fc&*0>— tTtt, gfS 
tf>:7 I— Af?9RI£r r50cmj , r l mj £V^ofcJ:5 

[0077] <m7mmm>%i<omsmm\^^ 

30 'tamioT^ V" — At — $ &m t 9^XZ>Zk&mMZ 
JS2-JR5 30IS«tB'CRWUfcJ:5^ WW-*!!** 

So 

[0078] mmmmxn, m^—^^M^^ 

A 9 0 J; oT^r#ofcfiI^ (GP St- 
40 Eim-^A^tfnO «^fllv^T. r(Dj:9ft^Ii:ft 

7WA»U 71 — ^©^IBHHbfcfrofc^ G 
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(57)Abstract: 

PROBLEM TO BE SOLVED: To efficiently generate and 
hold panorama images for regenerating a virtual space in 
building a walk-through virtual space, using IBR 
technique. 

SOLUTION: Plural pieces of frame data are generated 
and held, based on image data obtained by 
photographing, while moving plural image pickup devices 
to photograph in different directions. From the plural 
thus held pieces of frame data, a frame data group 
comprising frame data determined as being photographed 
at the same position are extracted as a frame data 
group comprising continuous frame data (S101-110). For 
the extracted frame data, the frame data are deleted, so 
that one piece of frame data exists for one position 
(S106, S1 1 1). In this way, in storing the frame data 
corresponding to the plural positions in a map, thinned- 
out frame data are stored to correspond to position in 
the map. 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] A maintenance means to hold two or more frame data obtained from the image data 
inputted while moving two or more locations, An extract means to extract the frame data judged 
to be what photoed in the same location more nearly substantially than two or more frame data 
held at said maintenance means, The image processing system characterized by having a 
deletion means to delete the overlapping frame data which were extracted with said extract 
means, and a storing means to match and store in the location on a map the frame data which 
remained after activation of said deletion means. 

[Claim 2] It is the image processing system according to claim 1 which the location data which 
show each of two or more of said frame data the location at the time of photography match said 
maintenance means, is held, and is characterized by said extract means extracting the frame 
data whose location data correspond. 

[Claim 3] Said location data are an image processing system according to claim 2 characterized 
by being obtained by GPS. 

[Claim 4] It is the image processing system according to claim 2 which photos said image data, 
moving two or more image pick-up equipments which photo a different direction, and is 
characterized by obtaining said location data from the range finder which measures the travel of 
the mobile which carries said two or more image pick-up equipments, and moves. 
[Claim 5] It is the image processing system according to claim 1 which the rate data obtained by 
each of two or more of said frame data at the time of photography match said maintenance 
means, is held, and is characterized by said extract means extracting the frame data 
constellation which said rate data become from ** staple fiber REMU data about a halt. 
[Claim 6] It is the image processing system according to claim 1 which is further equipped with a 
judgment means by which they judge whether it is the thing of the same location, by the 
comparison of two frame data, and is characterized by said extract means extracting the frame 
data judged by said judgment means to be the thing of the same location. 
[Claim 7] Said judgment means is an image processing system according to claim 6 
characterized by computing the least square error between two frame data, and judging the two 
frame data concerned to be the thing of the same location when the computed least square 
error is below a predetermined value. 

[Claim 8] said judgment means — the corresponding points of two frame data — being based — 
this — the image processing system according to claim 6 characterized by being characterized 
by computing the relative distance between two frame data, and judging the two frame data 
concerned to be the thing of the same location when this relative distance is below a 
predetermined value. 

[Claim 9] The frame data which photo said image data, respectively, moving two or more image 
pick-up equipments which photo a different direction, and are adopted in said judgment means 
are an image processing system according to claim 6 to 8 characterized by being obtained from 
sideways image pick-up equipment to the migration direction among said two or more image 
pick-up equipments. 

[Claim 10] It is the image processing system according to claim 2 which is further equipped with 
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a setting-out means to set up the sampling period of frame data, and is characterized by said 
extract means extracting frame data with a distance smaller than the distance equivalent to the 
sampling period set up with said setting-out means acquired with location data. 
[Claim 1 1] An acquisition means to acquire the frame data constellation photoed along the 
setting-out means [ to set up the sampling period of frame data ], and line top between two on a 
map, So that it may become the number of frame data computed with a calculation means to 
compute the number of frame data which should exist on the segment for these two points, and 
said calculation means based on the distance and said sampling period for said two points The 
image processing system according to claim 1 characterized by having further an infanticide 
means to perform infanticide to two or more frame data acquired with said acquisition means. 
[Claim 1 2] The extract process which extracts the frame data judged to be what photoed in the 
same location more nearly substantially than the maintenance means holding two or more frame 
data obtained from the image data obtained by two or more image pick-up equipments, The 
image-processing approach characterized by having the deletion process which deletes the 
overlapping frame data which were extracted at said extract process, and the storing process 
which matches and stores in the location on a map the frame data which remained after 
activation of said deletion process. 

[Claim 1 3] It is the image-processing approach according to claim 1 2 which the location data in 
which the location at the time of photography is shown match, are held at each of two or more 
of said frame data at said maintenance means, and is characterized by said extract process 
extracting the frame data whose location data correspond. 

[Claim 14] Said location data are the image-processing approach according to claim 12 
characterized by being obtained by GPS. 

[Claim 1 5] It is the image-processing approach according to claim 1 3 which photos said image 
data, respectively, moving two or more image pick-up equipments which photo a different 
direction, and is characterized by obtaining said location data from the range finder which 
measures the travel of the mobile which carries said two or more image pick-up equipments, and 
moves. 

[Claim 1 6] It is the image-processing approach according to claim 12 that the rate data obtained 
at the time of photography match, and are held at each of two or more of said frame data at said 
maintenance means, and it is characterized by said extract process extracting the frame data 
which said rate data show a halt. 

[Claim 1 7] It is the image-processing approach according to claim 1 2 which is further equipped 
with the judgment process they judge whether it is the thing of the same location to be by the 
comparison of two frame data, and is characterized by said extract process extracting the frame 
data judged according to said judgment process to be the thing of the same location. 
[Claim 18] Said judgment process is the image-processing approach according to claim 17 
characterized by computing the least square error between two frame data, and judging the two 
frame data concerned to be the thing of the same location when the computed least square 
error is below a predetermined value. 

[Claim 19] said judgment process — the corresponding points of two frame data — being based 
— this — the image-processing approach according to claim 1 7 characterized by being 
characterized by computing the relative distance between two frame data, and judging the two 
frame data concerned to be the thing of the same location when this relative distance is below a 
predetermined value. 

[Claim 20] The frame data which photo said image data, respectively, moving two or more image 
pick-up equipments which photo a different direction, and are adopted in said judgment process 
are the image-processing approach according to claim 1 7 to 1 9 characterized by being obtained 
from sideways image pick-up equipment to the migration direction among said two or more image 
pick-up equipments. 

[Claim 21] It is the image-processing approach according to claim 13 which is further equipped 
with the setting-out process which sets up the sampling period of frame data, and is 
characterized by said extract process extracting frame data with a distance smaller than the 
distance equivalent to the sampling period set up at said setting-out process acquired with 
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location data. 

[Claim 22] The acquisition process which acquires the frame data constellation photoed along 
the line top between the setting-out process which sets up the sampling period of frame data, 
and 2 on a map. So that it may become the number of frame data computed at the calculation 
process which computes the number of frame data which should exist on the segment for these 
two points, and said calculation process based on the distance and said sampling period for said 
two points The image-processing approach according to claim 12 characterized by having further 
the infanticide process which performs infanticide to the frame data constellation acquired at 
said acquisition process. 

[Claim 23] The storage characterized by storing a control program for a computer realizing the 
image-processing approach according to claim 1 2 to 22. 

[Claim 24] A control program for a computer to realize the image-processing approach according 
to claim 1 2 to 22. 



[Translation done.] 
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1 This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the image processing system which expresses a 
virtual space based on the image data obtained by photography of real space, and its approach. 
[0002] 

[Description of the Prior Art] With the photography equipment carried in the mobile, photo actual 
space and based on the photoed on-the-spot photo image data the attempt which expresses the 
photoed actual space as a virtual space using a computer is proposed (for example, Endo — ) 
Katayama, Tamura, Hirose, Watanabe, and mountain stream: "cyber- conversion to signals of 
urban space using a migration vehicle loading camera" (a **** society — ) PA— 3-4, pp.276 1997 
[ -277, 1 or Hirose, Refer to Watanabe, mountain stream, Endo, Katayama, and Tamura: 
"generation - of a broader-based virtual space using the construction (2)-on-the-spot photo 
image of cyber- image urban space using a migration vehicle loading camera" (the 2nd time 
convention collected works of the Virtual Reality Society of Japan, pp.67 1997 [ -70 or]) etc. 
[0003] Although the technique which reproduces the geometric geometric model of actual space 
based on on-the-spot photo image data based on the on-the-spot photo image data photoed by 
the photography equipment carried in the mobile, and is expressed with the conventional CG 
technique as the technique of expressing the photoed actual space as a virtual space on it is 
mentioned, there is a limitation in respect of the accuracy of a model, precision, reality nature, 
etc. On the other hand, the Image-Based Rendering (IBR) technique which expresses a virtual 
space using an on-the-spot photo image attracts attention in recent years, without performing 
the rendering using a model. An IBR technique is a technique which generates the image seen 
from the view of arbitration based on two or more on-the-spot photo images. Since the IBR 
technique is based on the on-the-spot photo image, the expression of a realistic virtual space is 
possible for it 

[0004] In order to build the virtual space in which walk-through is possible using such an IBR 
technique, it is necessary to perform generation and presentation of the image according to the 
location in an experience person's virtual space. Therefore, in this kind of system, each frame of 
on-the-spot photo image data and the location in a virtual space are matched and saved, the 
frame which corresponds based on an experience person's location and eye direction in a virtual 
space is acquired, and this is reproduced. 

[0005] It is common to use the positioning system using the satellite represented as the 
technique of obtaining the location data in actual space by GPS (Global Positioning System) 
used for the car-navigation system etc. The technique matched using a time code as the 
technique of matching the location data obtained from GPS etc. and on-the-spot photo image 
data is proposed (JP.1 1-1 68754 ,A). By this technique, matching with each frame of on-the-spot 
photo image data and location data is performed by matching the time-of-day data contained in 
location data, and the time code added to each frame of on-the-spot photo image data. 
[0006] An experience person enables it to see the direction of desired in walk-through in such a 
virtual space in each view location. For this reason, the image of each view location is saved by 
the panorama image which covers the range larger than the field angle at the time of playback, 



http://www4.ipdl.ncipi.gojp/cgi-bin/tran_web_cgi_eije 



2006/06/20 



JP.2002-269592.A [DETAILED DESCRIPTION] 



2/10 



the partial image which should be reproduced from a panorama image based on the view location 
and the direction of a look in a virtual space of an experience person is cut down, and it is 
possible to display this. 
[0007] 

[Problem(s) to be Solved by the Invention] However, when urban space is photoed with the 
photography equipment carried in this, using a vehicle as a mobile, the image data of the same 
point will be physically photoed by the situation of a signal, the congestion situation of traffic, 
etc. over a multiple frame. For this reason, the collected on-the-spot photo image data will 
contain many panorama images of the same part. In case this causes the duplication of data 
needless to say, for example, the database of the panorama image for walk-through playback is 
built a useless panorama image will be accumulated. Naturally, it leads also to useless 
consumption of storage resources, such as a hard disk 

[0008] This invention is made in view of the above-mentioned technical problem, is set to build 
the virtual space in which walk-through is possible using an IBR technique, and it aims at 
enabling generation and maintenance of the panorama image for virtual space playback 
efficiently. 
[0009] 

[Means for Solving the Problem] The picture reproducer by this invention for attaining the 
above-mentioned object is equipped with the following configurations. Namely, a maintenance 
means to hold two or more frame data obtained from the image data inputted while moving two 
or more locations, An extract means to extract the frame data judged to be what photoed in the 
same location more nearly substantially than two or more frame data held at said maintenance 
means, It has a deletion means to delete the overlapping frame data which were extracted with 
said extract means, and a storing means to match and store in the location on a map the frame 
data which remained after activation of said deletion means. 

[0010] Moreover, the image reconstruction approach by this invention for attaining the above- 
mentioned object is equipped with the following processes. That is, it has the extract process 
which extracts the frame data judged to be what photoed in the same location substantially than 
the maintenance means holding two or more frame data obtained from the image data obtained 
by two or more image pick-up equipments, the deletion process which delete the frame data 
which were extracted at said extract process, and overlap, and the storing process which match 
with the location on a map the frame data which remained after activation of said deletion 
process, and store. 
[0011] 

[Embodiment of the Invention] Hereafter, the suitable operation gestalt of this invention is 
explained, referring to an accompanying drawing. 

[001 2] <the 1 st operation gestalt> — the walk-through system of the virtual space by this 
operation gestalt is explained first. With this operation gestalt, panorama image data is generated 
from the on-the-spot photo image data photoed and obtained by two or more photography 
equipments carried, for example in mobiles, such as an automobile, and this panorama image data 
is matched with the map data as a location of actual space, and is held. And walk-through in a 
virtual space is realized by generating a display image from the panorama image data currently 
held according to the view location and the direction of a look in a virtual space of an experience 
person. 

[001 3] Drawing 1 is a block diagram explaining the functional configuration of the walk-through 
system by this operation gestalt. This walk-through system is constituted including the image 
data collecting system 90 and picture reproducer 1. Picture reproducer 1 has the image data 
storage section 10, the map data storage section 20, an image and the map matching section 30, 
a control unit 40, the matching data storage section 50, the image reconstruction control section 
60, and a display 70. 

[0014] The image data storage section 10 stores the frame data as on-the-spot photo image 
data obtained by the image data collecting system 90 explained in full detail later. The map data 
storage section 20 stores map data including the coordinate information for expressing each 
location on the image information of a map, and this map image with the coordinate relevant to 
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LONG and the LAT. A photograph is taken at least with the image data collecting system 90 by 
the map data storage section 20, and the map data of the range corresponding to the actual 
spatial position of the frame data saved in the image data storage section 10 are saved in it In 
addition, map data are saved at non-illustrated a hard disk, RAM, or other external storage. 
[001 5] An image and the map matching section 30 generate the panorama image data of each 
view location from the frame data saved in the image data storage section 1 0, and matches this 
with the map data saved in the map data storage section 20. In this way, the panorama image 
data and map data which were matched are saved in the matching data storage section 50 as 
matching data. In addition, in an image and the map matching section 30, a panorama image is 
generated from the frame data obtained from two or more photography equipments in the same 
time of day, the map data (location data on a map) which correspond from the GPS information 
in the time of day are specified, and these are saved in matching and the matching data storage 
section 50. A time code is added to each of the frame data obtained from GPS information and 
two or more photography equipments, and the frame of the same time of day and GPS 
information are acquired by this time code so that it may mention later. 

[0016] A control unit 40 is equipped with a mouse, a keyboard, a joy stick, etc. In addition, in 
above-mentioned image and map matching section 30, it is good also as edit of matching with 
the frame saved in the image data storage section 10 according to the actuation input from a 
control unit 40 and the map data saved in the map data storage section 20 being possible. 
[0017] The image reconstruction control section 60 determines an experience person's view 
location (location on a map), and the direction of a look according to the actuation input from a 
control unit 40, reads required image data from the data saved in the matching data storage 
section 50, and generates the image data for displaying by the display 70. 

[001 8] Drawing 2 is drawing showing the example of a configuration of the image data collecting 
system 90 for collecting the frame data saved in the image data storage section 10. As shown in 
drawing 2 , this image data collecting system 90 is divided into three parts, the photography 
section 91 , the Records Department 92, and the A/D-conversion section 93. Hereafter, each 
part is explained to a detail with reference to drawing 3 - drawing 5 . 

[0019] Drawing 3 is the block diagram showing the configuration of the photography section 91 in 
a detail. The photography section 91 is carried and used for mobiles, such as a vehicle, with this 
operation gestalt. The photography section 91 consists of n sets (n>=1) (91-1 - n) of video 
cameras, and the synchronizing signal generating section 94. n sets of the photography timing of 
a video camera 91-1 - n are made in agreement using the external synchronizing signal possible 
[ into a video camera 91-1 - n /, respectively / the external synchronizing signal from the 
synchronizing signal generating section 94 ] and outputted from the synchronizing signal 
generating section 94 with this operation gestalt. 

[0020] Drawing 4 is the block diagram showing the configuration of the Records Department 92 
in a detail. The Records Department 92 has the time code generating section 95, the image 
transcription section (this example the video cassette recorder VCR) 92-1 corresponding to a 
video camera 91-1 - n - n. The image output from n sets of the video cameras 91-1 of the 
photography section 91 - and n turns into an input of the image transcription section 92-1 - n, 
respectively. Moreover, the time code generating section 95 supplies the time code showing 
photography time of day to each VCR 92-1 - n. VCR 92-1 - n incorporate the corresponding 
time code from the image input and the time code generating section 95 from a video camera, 
and record it as a video data with a time code. 

[0021] The image information collected by each of VCR 92-1 - n as mentioned above is changed 
into digital image data by the A/D-conversion section 93, and is saved in the image data storage 
section 10. Drawing 5 is the block diagram showing the configuration of the A/D-conversion 
section 93 in a detail. The A/D-conversion section 93 has a personal computer (henceforth, PC) 
96, the video capture board (henceforth, capture board) 93-1 corresponding to each of VCR 92- 
1 - n - n. A capture board is not necessarily required by the number of a video camera, and a 
number of capture boards which can be carried in PC may be shared. The A/D-conversion 
section 93 changes into digital image data (for example, AVI format) the analog image data 
supplied from each VCR, and saves it at the storage of the image data storage section 10 which 
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was connected to PC96 and which is constituted including a hard disk etc., or others. 
[0022] Furthermore, in this system, to the timing which a time code generates from the time 
code generating section 95, location data are acquired, and the acquired location data are 
matched with the time code at the event, and are held from GPS97. 

[0023] Drawing 6 is drawing showing the example of a storing condition of a video data and 
location data (GPS measurement result data) in the image data storage section 10. As shown in 
drawing 6 , the frame to which the time code was added, and the location data with which the 
time code was added will be stored in the image data storage section 1 0. Therefore, a frame and 
location data are matched through this time code. In addition, although drawing 6 shows only the 
video data from one video camera, as the video data was mentioned above, the amount of [ of a 
video camera ] (n sets) number exists. 

[0024] In addition, calculation of the three-dimension location in GPS is well-known, and omits 
detailed explanation here. The LAT and the LONG data P (theta, phi) which are obtained serially 
are matched and stored in the time code from the time code generating section 72 by GPS, 
photoing an image by n sets of video cameras as mentioned above. Therefore, the LAT and 
LONG data which were obtained from GPS can be matched with each frame of a video data 
through this time code. 

[0025] In addition, in case image data is incorporated in PC96 through the capture board 93-1 - 
n, a capture initiation part and a termination part are determined using the time code currently 
recorded, and all the digital image data acquired through n sets of VCR and a capture board 
consist of capture data of the same die length photoed at this time of day. 

[0026] Next, picture reproducer 1 is explained. Drawing 7 is the block diagram showing the hard 
configuration of the picture reproducer 1 by this operation gestalt The hard configuration shown 
in drawing 7 is equivalent to the configuration of the usual personal computer. In drawing 7 , a 
disk 105 constitutes the image data storage section 10, and the frame data and location data 
which were obtained by the image data collecting system 90 explained in relation to drawing 2 - 
drawing 6 are memorized. In addition, a disk 1 05 constitutes not only the above-mentioned image 
data storage section 1 0 but the map data storage section 20 and the matching data storage 
section 50 which were shown in drawing 1 . 

[0027] CPU101 functions as the image and the map matching section 30 which matches and 
stores image data and map data, and an image reconstruction control section 60 which performs 
image reconstruction based on the image data saved in the matching data storage section 50 by 
performing the program saved at a disk 105, ROM106, or external storage (un-illustrating). 
[0028] When CPU101 performs various kinds of display directions to the display controller 102, 
the display of the request to a drop 104 is made by the display controller 102 and the frame 
buffer 1 03. In addition, although CRTC was shown as a display controller 1 02 and CRT was 
shown as a drop 104 by a diagram, of course as a drop, not only a cathode-ray tube but a liquid 
crystal display etc. may be used. In addition, CRTC102, a frame buffer 103, and CRT104 
constitute the above-mentioned display 70. A mouse 108, a keyboard 109, and a joy stick 110 
are for performing the actuation input of the user to image maintenance and the regenerative 
apparatus 1 concerned, and constitute ** Li and the above-mentioned control unit 40. 
[0029] Next, the outline of the actuation of picture reproducer 1 in the walk-through system of 
this operation gestalt equipped with the above configuration is explained. Drawing 8 is drawing 
explaining the content of processing of picture reproducer 1 in the walk-through system of this 
operation gestalt 

[0030] The location data with a time code based on the frame data with a time code based on 
the video data obtained by n sets of video cameras 91-1 - and n with the image data collecting 
system 90 mentioned above and the location data obtained by GPS97 are stored in the image 
data storage section 1 0. 

[0031] An image and the map matching section 30 change the location data corresponding to the 
time code concerned into the location on a map with reference to the map data currently held at 
the map data storage section 20 while it joins the frame data of the same time code and 
generates a panorama image. And the matching data 210 which matched the obtained panorama 
image and the location on a map are generated, and it stores in the matching data storage 
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section 50. 

[0032] In addition, in the matching data storage section 50, the following data storage gestalt is 
taken with this operation gestalt. That is, a crossing and a corner of a street are made into a 
dividing point, ID is assigned to each dividing point and a path by making into a path the segment 
inserted by the dividing point and the dividing point, and this ID is added to a corresponding 
frame. A number is shaken at the frame group corresponding to one path sequentially from a 
head. 

[0033] Drawing 9 is drawing explaining this situation. ID called R1 is given to the segment 
inserted into the dividing point whose ID is C1, and the dividing point which is C2 in drawing 9 . 
The response with these ID and a map is saved in the map data storage section 20. 
[0034] When the frame corresponding to dividing points C1 and C2 is specified based on GPS 
data etc., the frame group inserted into these frames will correspond to a path R1 . By a diagram, 
n frames exist in this frame group. And ID of C1 and C2 is given to the frame corresponding to 
dividing points C1 and C2, respectively, and R1-1 - R1-n is given to each frame of a frame group 
in order. 

[0035] In addition, although matching with a dividing point and a frame shall be automatically 
performed according to GPS data, while a user plays video, it is also possible to match by 
specifying the crossing on a frame and a corresponding map. In this case, the thing which assign 
at equal intervals on the segment which connects the dividing point concerned, for example (the 
segment which connects C1 and C2 is equally divided into n+1, and n frames are assigned to 
each division location in order in the above-mentioned example) and for which a system is 
constituted, without using GPS if it is made like is possible for the location of each frame of the 
frame group inserted into the dividing point. 

[0036] Walk-through playback is performed using the matching data stored as mentioned above. 
If walk-through actuation using joy stick 110 grade is performed from a control unit 40, according 
to this, the view location (location in the path on a map) and the direction of a look of [ on an 
experience person's map ] will be generated. The image reconstruction control section 60 
acquires the image which should be displayed on a display 70 from the matching data 21 0 based 
on this generated view location, the direction of a look, and the field angle displayed on a display 
70, and displays this on a display 70. For example, when having turned to the direction of 15 
degrees at a point on a map, the partial image corresponding to the direction of 15 degrees in 
the panorama image corresponding to a point will be extracted. And walk-through will be realized 
when the image reconstruction control section 60 performs acquisition of a display image, and 
the display to a display 70 as mentioned above about the view location and the direction of a 
look by which a sequential decision is made according to an experience person's location moving 
in a map top. 

[0037] In the above system, the image data collecting system 90 collects image data by taking a 
photograph by carrying the photography section 91 which has two or more cameras in mobiles, 
such as a vehicle. Therefore, it is impossible first for a photograph to be taken running a general 
path, and to take a photograph, moving at a low speed. Especially, the image data in which the 
unnecessary frame which the content of photography overlapped was contained will be collected 
by a halt by the signal, and low-speed migration by traffic congestion. With this operation gestalt, 
such a useless frame is eliminated and generation and maintenance of panorama data are 
enabled efficiently. 

[0038] The image and the map matching section 30 of this operation gestalt perform infanticide 
processing which thins out the unnecessary frame which the camera station overlapped first 
among the frames saved in the image data storage section 10. And after being culled out, a 
panorama image is generated using the frame which remained, and it saves in the matching data 
storage section 50. Hereafter, the infanticide processing by the 1st operation gestalt is 
explained. 

[0039] In infanticide processing, it is collected by the image data collecting system 90, and 
investigates whether the adjoining frame is the image of the same location about the frame 
obtained from one video camera in the image data saved in the image data storage section 10, 
and when the same, the frame showing the image of the same adjoining location is deleted. 
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[0040] Drawing 10 is a flow chart explaining the infanticide processing by the 1st operation 
gestalt. In step S101, i which points out the location data for a check (henceforth GPS data) is 
first set as initial value "1 ." Next in step S1 02, the i-th GPS data Pi are acquired from the 
image data storage section 1 0 (that is, Pi is GPS data corresponding to the frame i 
corresponding to the hth time code). Moreover, in step S103, j which points out the GPS data 
set as the object of comparison with this GPS data Pi is set to "rM", i.e., the GPS data 
corresponding to the frame j of the next door of Frame i. 

[0041] At step S104, Frame j and the corresponding GPS data (namely, j-th GPS data) Pj are 
acquired from the image data storage section 10, and step S105 compares Pi and Pj. And when 
both are equal (Pi=Pj), it progresses to step S106, and marking is carried out to the j-th time 
code or the j-th GPS data in order to delete Frame j. And it is step S107, one j is added in order 
to investigate the location of the next frame further, and with [ in step S108 / j ] n [ below ], it 
returns to step S104. It is the frame group which continues from Frame i, and a mark will be 
attached by the above processing in order to delete the frame group which consists of a frame 
photoed in the same location as Frame i. For example, when the frame photoed in the same 
location is carrying out 5 (referred to as 11-15) continuation, a delete mark will be given to the 
GPS data or the time code which corresponds so that the GPS data of these Five frames may 
become equal and it may delete 12-15. In addition, since it means finishing examination about all 
frames when j becomes larger than n (frame number of a processing object) at step S108, 
processing is advanced to step S1 1 1 . 

[0042] On the other hand, in step S105, when Pi and Pj are not equal, it progresses to step S109 
and the value of i is assigned to j. And if it is not i=n at step S1 10, it will return to step S102. In 
this way, when it investigates whether the frame group which consists of a continuous frame 
photoed in the same location exists and exists in all n frames, marking can be performed in order 
to eliminate the overlapping frame. 

[0043] Next, in step S1 1 1 , all the frames corresponding to the GPS data (or time code) with 
which marking was performed are deleted. In this way, the image duplication of data of the same 
location can be eliminated about the frame sequence acquired from all cameras. 
[0044] In this way, after the overlapping frame is deleted, an image and the map matching 
section 30 generate a panorama image using the left-behind frame, matches this with the data in 
which the location on a map is shown, and saves it in the matching data storage section 50. 
[0045] in addition, a ****** [ that location data are in agreement in decision of being the thing of 
the same location in the above-mentioned processing ] — having carried out (step S105) — 
when it is in the range with both difference (distance), you may make it consider that it is the 
same, even if not strictly in agreement in consideration of the measurement error by GPS etc. In 
this case, what is necessary is just to judge whether fCPi, Pj) <alpha is satisfied in step S105. 
Here, fQ is a function which converts into a distance value the GPS data expressed with the 
LAT and LONG. In addition, since the operation approach itself is obvious to this contractor, 
explanation is omitted. If it does in this way, the continuous frame group in the range of a 
location alpha can be represented with one frame of the head. 

[0046] As mentioned above, since according to the 1 st operation gestalt it leaves one frame (top 
frame) and other data were deleted based on the GPS data matched with the frame about the 
continuous frame group consider that a photograph was taken in the same location, useless 
processing can be prevented in generation of a panorama image. 

[0047] With the 1st operation gestalt of the <operation gestalt of 2nd> above, it culled out by 
judging chiefly whether it is photography data of the same location according to GPS data. With 
the 2nd operation gestalt, it judges whether it is photography data of the same location using 
rate data. 

[0048] In addition, with the 2nd operation gestalt, in the A/D-conversion section 93 of the image 
data collecting system 90 shown in drawing 5 , a speedometer is connected instead of GPS97 
and the measurement value is recorded synchronizing with generating of the time code by the 
time code generating section 95. 

[0049] Drawing 1 1 is a flow chart explaining the infanticide processing by the 2nd operation 
gestalt. In step S201 , i which points out the rate data for a check is first set as initial value "1 ." 
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Next, in step S202, the i-th rate data (namely, rate data corresponding to the i-th time code) Vi 
are acquired from the image data storage section 10. Moreover, in step S203, j which points out 
the rate data set as the object of comparison with this rate data Vi is set to "i+1 ", i.e., the rate 
data corresponding to the next frame. [ Vj ] 

[0050] At step S204, the rate data Vj matched with Frame j are acquired from the image data 
storage section 10, and it is confirmed whether both Vi and Vj are 0 at step S205. And when 
both both are 0 (Vi=Vj=0), it progresses to step S206, and marking is carried out to the rate data 
Vj or the j-th time code in order to delete Frame j. And it is step S207, one j is added in order to 
investigate the rate data in the next frame further, and with [ in step S208 / j ] n [ below 1 it 
returns to step S204. 

[0051] With the frame which 0 follows [ rate data ], since migration of a camera has stopped by a 
certain cause, the frame of the same location will overlap and exist. Therefore, duplication is 
eliminated by leaving only the first frame and deleting other frames by the above-mentioned 
processing. For example, when the frame of 0 is carrying out [ rate data ] 5 (referred to as 11-15) 
continuation, a delete mark will be given to 12-15. In addition, since it means finishing examination 
about all frames when j becomes larger than n (frame number of a processing object) at step 
S208, processing is advanced to step S21 1 . 

[0052] On the other hand, when not set to Vi=Vj=0 in step S205, it progresses to step S209 and 
the value of i is assigned to j. And if it is not i=n at step S210, it will return to step S202. In this 
way, marking of ** which detects the continuous frame photoed in the same location about all n 
frames, and deletes the overlapping frame can be performed. 

[0053] Next, in step S21 1, all the frames corresponding to the rate data or the time code with 
which marking was performed are deleted. In this way, the image duplication of data of the same 
location can be eliminated about the frame sequence acquired from all cameras. 
[0054] As mentioned above, according to the 2nd operation gestalt, it becomes possible to 
detect and delete the overlapping image data using the passing speed which formed the speed 
indicator in mobiles, such as a vehicle, and was measured. 

[0055] In addition, you may use together with the configuration using the GPS data explained 
with the 1 st operation gestalt. Since data acquisition may become impossible with geography 
etc., GPS can also be constituted so that infanticide processing using rate data may be 
performed in the part from which GPS data were not obtained. 

[0056] Moreover, since GPS data may include an error, the detection precision of the 
overlapping frame is used together with rate data in order to improve more, and you may make it 
delete a frame. What is necessary is just to make applicable to deletion the frame judged that is 
the same location from both GPS data and rate data in this case. 

[0057] The <3rd operation gestalt> Although the rate sensor was used with the 2nd operation 
gestalt, the vehicle carrying a camera is equipped with a range finder, and the mileage of the 
vehicle concerned is recorded synchronizing with a time code, and the measurement data 
detects the same frame group as a meeting of the frame of the same part, and you may make it, 
delete the frame which overlaps about the frame group concerned in addition. 
[0058] In this case, the frame which overlaps Pi and the distance measurement value of Frame j 
with the same procedure as Pj, then the 1 st operation gestalt in the distance measurement value 
of Frame i can be detected and deleted. 

[0059] Moreover, when it is in the range with both difference, you may make it consider that it is 
the same by judging whether |Pi-Pj|<alpha is satisfied in step S105, like [ when a range finder is 
used ] the 1 st operation gestalt, even if not strictly in agreement In addition, what is necessary 
is just to take the difference of a measurement value simply by the above-mentioned formula, 
since a distance measurement value is used as Pi and Pj with the 3rd operation gestalt. 
[0060] With the 4th operation gestalt of the <4th operation gestalt>, the frame which does not 
use GPS data or rate data, either but overlaps from the content of the adjoining image is 
detected, and this is deleted. 

[0061] Drawing 12 is a flow chart explaining the infanticide processing by the 4th operation 
gestalt However, the following processings are performed about the frame sequence acquired 
from one of two or more cameras which the image data collection section 90 owns. In step S301, 
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i which points out the frame for a check is first set as initial value "1." Next, in step S302, j 
which points out the frame set as the object of comparison with this frame i is set to "i+1 ", i.e., 
the next image. 

[0062] At step S303, the least square error E between both the images of Frame i and Frame j is 
calculated, and it judges whether this error E is smaller than a predetermined threshold by step 
S304. And when the value of Error E is smaller than a threshold, it is judged that Frames i and j 
are the images of the same location, and marking is carried out in order to delete Frame j from 
the image sequence concerned at step S305. And at step S306, one j is added in order to 
investigate the following frame, and with [ in step S307 / j ] n [ below ], it returns to step S304. 
[0063] On the other hand, when Error E is beyond a threshold in step S304, it is judged with the 
image expressed by Frames i and j not being the same (it not being the image of the same 
location), and progresses to step S308, and the value of i is assigned to j. And if it is not i=n at 
step S309, it will return to step S302. In this way, marking of ** which detects the continuous 
frame photoed in the same location about all n frames, and deletes the overlapping frame can be 
performed. 

[0064] Next, the frame to which marking was performed is deleted in step S310. And in step 
S31 1 , the frame of the frame number (or time code) same also about the frame sequence 
acquired from other cameras as the frame deleted at step S310 is deleted. 

[0065] In addition, although the frame used in the 4th operation gestalt uses the image obtained 
from one of n cameras shown in drawing 3 , when the camera is carried in a vehicle, it is 
desirable [ a camera ] to turn to the include angle of about 90 degrees to a travelling direction. It 
is because the existence of migration tends to appear on a screen. 

[0066] Moreover, although the frame from one camera was used with the above-mentioned 
operation gestalt, it may be made to perform the above-mentioned processing about two or 
more frame sequences. For example, processing which showed the longitudinal direction to 
drawing 10 about each of two frame sequences acquired from two suitable cameras is performed 
to a travelling direction, and when marking of deletion is performed to both frames which 
correspond between both-system trains, the frame of a frame number is deleted. By carrying out 
like this, even when change of one near scene of a travelling direction is scarce also in vehicle 
migration, it can judge whether it is an object for deletion surely (for example, when it is the sea). 

[0067] With the 4th operation gestalt of the <5th operation gestalt>, the least square error was 
used in the judgment (steps S303 and S304) of whether to be the same image. However, decision 
whether two images are the same is not restricted to such technique. The 5th operation gestalt 
explains the case where it carries out as an example of a procedure which judges whether two 
images are photoed from the same location by asking for the relative position of both images 
from the corresponding points in an image. 

[0068] Drawing 1 3 is a flow chart explaining the infanticide processing by the 5th operation 
gestalt. However, the following processings are performed about the frame sequence acquired 
from one of two or more cameras which the image data collection section 90 owns. In step S401, 
i which points out the frame for a check is first set as initial value "t." Next, in step S402, j 
which points out the frame set as the object of comparison with this frame i is set to "i+1 ", i.e., 
the next image. 

[0069] At step S403, the corresponding points in both the images of Frame i and Frame j are 
extracted, and the relative physical relationship of both images is searched for from the 
extracted corresponding points. In addition, to this contractor, since the extract of the 
corresponding points from an image and the calculation of a location based on corresponding 
points are common knowledge, they omit explanation. 

[0070] At step S404, the relative distance between both images is found from the relative 
position for which it asked at step S403, and it judges whether this relative distance is smaller 
than a predetermined threshold. And when a relative distance is smaller than a threshold, it is 
judged that Frames i and j are the images of the same location, and marking is carried out in 
order to delete Frame j from the image sequence concerned at step S405. And at step S406, 
one j is added in order to investigate the following frame, and with [ in step S407 / j ] n [ below ], 
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it returns to step S404. 

[0071] On the other hand, when a relative distance is beyond a threshold in step S404, it is 
judged with the image expressed by Frames i and j not being the same (it not being the image of 
the same location), and progresses to step S408, and the value of i is assigned to j. And if it is 
not i=n at step S409, it will return to step S402. In this way, marking of ** which detects the 
continuous frame photoed in the same location about all n frames, and deletes the overlapping 
frame can be performed. 

[0072] Next the frame to which marking was performed is deleted in step S410. And in step 
S41 1, the frame of the frame number same also about the frame sequence acquired from other 
cameras as the frame deleted at step S410 or a time code is deleted. 

[0073] With the <6th operation gestalt> above 1 st - the 5th operation gestalt, the frame of the 
same location photoed while the vehicle as a mobile stopped was detected, by deleting this, the 
same frame was removed and effective use of a resource and the increase in efficiency of 
processing were attained. 

[0074] With the 6th operation gestalt, further, a frame is acquired at intervals of a request [ in / 
for a frame / real space ], and storing is made possible at the matching data storage section 50. 
Although this effectiveness described the purport you may make it judge whether it is the frame 
which is processing of step S105 in the 1st operation gestalt and the 3rd operation gestalt, and 
should be deleted using f(Pi. Pj) <alpha or |Pi-Pj|<alpha, it is acquired with constituting so that a 
user can set this alpha as a request. 

[0075] Thus, with constituting, spacing of a frame can be mostly made into the regular intervals 
of alpha, and spacing of the panorama image stored in the speed of the mobile at the time of 
image data collection at ******** anc ) the matching data storage section 50 can be mostly made 
regularity (alpha). Consequently, the database of the suitable panorama image for walk-through 
playback is obtained. 

[0076] In addition, what is necessary is just to give the distance of real space as an example of 
the specification method of alpha, as it said that desired frame spacing was "50cm" and "1m." 
[0077] The <7th operation gestalt> In the 1 st operation gestalt, also when it did not have GPS 
data, by the dividing point and path on a map, frame data could be assigned and purport 
explanation was given. Furthermore, even when it does not have GPSfdata, as the 2nd - the 5th 
operation gestalt explained, duplication of the video data based on a halt of the mobile under 
image data collection can be eliminated. However, even if it performs such processing, by the 
acceleration and deceleration of a mobile, the roughness and fineness of a frame will remain and 
a useless frame will exist in a part dense beyond the need. 

[0078] Although such a useless frame was eliminated and regulaMntervals-ization of a frame 
was performed with the 6th operation gestalt using the information on the location which took a 
photograph with the image data collecting system 90 (measurement value by GPS data and the 
range finder), it cannot apply, when it is a system without GPS and a range finder. With the 7th 
operation gestalt, when it does not have GPS and a range finder, the unevenness of the sampling 
period of a frame is absorbed. However, matching with a map which was explained by drawing 9 
shall be made. 

[0079] Drawing 14 is a flow chart explaining infanticide processing of the frame by the 7th 
operation gestalt In addition, it is desirable to remove the duplicate frame in advance of this 
processing by the technique explained with the 2nd operation gestalt and the 4th and 5th 
operation gestalt. 

[0080] First, in step S501 , the desired sampling rate S (a sheet/pixel) is set up. 0.1 which a 
sampling rate S will be a frame number per pixel on a map image, for example, will be the inverse 
number if you want to consider as one frame every 1 0 pixels is set up as a sampling rate S. 
[0081] Next, it progresses to step S502 and the frame group which should cull out is specified by 
specifying a desired path. Assignment of a path may display a map image, and may direct it with 
a pointing device, and you may make it input Path ID from a keyboard etc. If a path is specified, 
the coordinate on the map image of the dividing point of the ends of the path specified by 
progressing to step S503 will be acquired, and the distance D between the 2 dividing points 
concerned (pixel) will be found from these. At step S504, it asks for the frame number of the 
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frame group currently assigned to the specified path, and this is set to F. For example, in the 
example of drawing 9 , n frames exist and it becomes F=n. 

[0082] The die length (distance D) of the specified path and the number of the frames which 
should exist in the specified path concerned with the set-up sampling rate S are expressed as 
DS. In processing of steps S506-S509, in order to make the frame number (F sheets) of the 
frame group corresponding to the specified path into DxS sheets, the image which should be 
deleted is chosen. In addition, in this processing, before performing processing after step S506, it 
judges whether it is F>DxS, and since it is premised on being F>DxS, if it is no, this processing 
will be ended. 

[0083] Hereafter, the processing after step S506 is explained. At step S506, the variable f used 
as a counter is set as initial value 1. Next, in step S507, it checks as a frame which adopts the 
frame of eye x (f-1) (F-1) (/(DxS -1)) watch. In addition, what integer—ized the result of an 
operation is used for the value of x 0M) (F-1) (/(DxS -1)). At step S508, 1 is added to f, and 
processing of steps S507 and S508 is repeated until it becomes f=DxS +1 (target number of 
sheets) (step S509). 

[0084] F frames corresponding to the specified path are thinned out to DxS frames by deleting 
the frame which is not checked in step S510 by the above processing since DxS checked frames 
are obtained. When processing other paths, return and the above-mentioned processing are 
repeated to step S51 1 step S501. When not performing edit beyond this, this processing is ended 
from step S51 1 . 

[0085] As mentioned above, since according to the 7th operation gestalt the panorama number 
of sheets per unit distance (per pixel) is set up and a frame is thinned out, the ununiformity of 
sample spacing at the time of image data collection is absorbable. Moreover, since it culls out 
about the frame group assigned to the path according to the sequence, even if there are no GPS 
data etc., it can cull out appropriately. 

[0086] In addition, it cannot be overemphasized by the object of this invention supplying the 
storage (or record medium) which recorded the program code of the software which realizes the 
function of the operation gestalt mentioned above to a system or equipment, and reading and 
performing the program code with which the computer (or CPU and MPU) of the system or 
equipment was stored in the storage that it is attained. In this case, the function of the 
operation gestalt which the program code itself by which reading appearance was carried out 
from the storage mentioned above will be realized, and the storage which memorized that 
program code will constitute this invention. Moreover, it cannot be overemphasized that it is 
contained also when the function of the operation gestalt which performed a part or all of 
processing that the operating system (OS) which is working on a computer is actual, based on 
directions of the program code, and the function of the operation gestalt mentioned above by 
performing the program code which the computer read is not only realized, but was mentioned 
above by the processing is realized. 

[0087] Furthermore, after the program code by which reading appearance was carried out from a 
storage is written in the memory with which the functional expansion unit connected to the 
functional expansion card inserted in the computer or a computer is equipped, it is needless to 
say in being contained also when the function of the operation gestalt which performed a part or 
all of processing that the CPU with which the functional expansion card and functional expansion 
unit are equipped based on directions of the program code is actual, and mentioned above by the 
processing is realized. 
[0088] 

[Effect of the Invention] As explained above, according to this invention, it sets to build the 
virtual space in which walk-through is possible using an IBR technique, and generation and 
maintenance are efficiently attained in the panorama image for virtual space playback etc. 
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DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is a block diagram explaining the functional configuration of the walk-through 
system by this operation gestalt. 

[Drawing 2] It is drawing showing the example of a configuration of the image data collecting 
system 90 for collecting the frame data saved in the image data storage section 10. 
[Drawing 3] It is the block diagram showing the configuration of the photography section 91 in a 
detail. 

[Drawing 4] It is the block diagram showing the configuration of the Records Department 92 in a 
detail. 

[Drawing 5] It is the block diagram showing the configuration of the A/D-conversion section 93 
in a detail. 

[Drawing 6] It is drawing showing the example of a storing condition of a video data and location 
data (GPS measurement result data) in the image data storage section 10. 

[Drawing 7] It is the block diagram showing the hard configuration of the picture reproducer 1 by 
this operation gestalt. 

[Drawing 8] It is drawing explaining the content of processing of picture reproducer 1 in the 
walk-through system of this operation gestalt. 

[Drawing 9] It is drawing explaining signs that frame data are matched with the dividing point on 
a map, and a path. 

[Drawing 10] It is a flow chart explaining the infanticide processing by the 1 st operation gestalt 
[Drawing 11] It is a flow chart explaining the infanticide processing by the 2nd operation gestalt 
, [Drawing 12] It is a flow chart explaining the infanticide processing by the 4th operation gestalt 
[Drawing 13] It is a flow chart explaining the infanticide processing by the 5th operation gestalt 
[Drawing 14] It is a flow chart explaining infanticide processing of the frame by the 7th operation 
gestalt 
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